The present invention comprises an improved method of motion compensated upconversion in a video image upconversion unit of the type that uses motion compensation to generate an interpolated video field using motion vectors. The improved method of the present invention selects a motion compensated pixel that will provide a sharp video image by comparing a calculated correlation value of pixels with a threshold value.
A moving video image is transmitted by a video transmitter as a sequence of frames or pictures. Each frame or picture may be coded individually, but is displayed sequentially at a video rate. Each video frame is made up of two video fields, an odd video field and an even video field. More specifically, an individual frame denoted by the letter xe2x80x9cAxe2x80x9d is made up of an odd field denoted by the letters xe2x80x9cAoxe2x80x9d and an even field denoted by the letters xe2x80x9cAe.xe2x80x9d
When capturing or recording a video sequence, either frames or fields may be captured. When both the odd field and the even field of a video frame are captured at the same time, the picture is said to be a xe2x80x9cprogressivexe2x80x9d picture. The odd fields and the even fields are not generally used to describe progressive pictures. Instead the individual video frames (e.g., frame A, frame B) are used to describe progressive pictures. Most movie film material is composed of progressive pictures.
When the odd field and the even field of a video frame are captured at different times, the picture is said to be an xe2x80x9cinterlacedxe2x80x9d picture. The two fields are not combined to be displayed at the same time. Each field is treated and displayed separately. Most television material is composed of interlaced pictures.
In order to achieve maximum efficiency during the transmission process, not every field or frame is transmitted. That is, some of the individual fields or frames are dropped and are not transmitted. The dropped fields or frames are recreated by the video receiver from information taken from the fields or frames that are transmitted.
For example, the dropped fields or frames can be recreated simply by repeating the previous field or frame. Alternatively, if the display is delayed, the next field or frame that follows the dropped field or frame can be used to take the place of the dropped field or frame. It is also possible to replace the dropped field or frame by averaging the neighboring fields or frames on each side of the dropped field or frame.
There are difficulties with these simple approaches. Repeating a previous field or frame (or using the next following field or frame) in place of the dropped field or frame causes the perceived image to be jerky even when small motions are depicted in the video image. Averaging the fields or frames causes the perceived image to be blurred even when moderate motions are depicted in the video image.
A well known method for recreating the dropped fields or frames is motion compensated interpolation. In motion compensated interpolation (also referred to as xe2x80x9cbidirectional predictionxe2x80x9d) a subsignal with a low temporal resolution (typically one half to one third of the frame rate) is coded and the full resolution signal is obtained by interpolation of the low resolution signal and the addition of a correction term. The signal to be reconstructed by interpolation is obtained by adding a correction term to a combination of a past and a future reference.
Video sequences exist in various formats. For example, high definition (HD) television video sequences may be displayed in any one of eighteen (18) different formats. The process of converting a video sequence from one format to another format is called xe2x80x9cscan rate conversion.xe2x80x9d
Scan rate conversion may be used to reduce image flicker in televison images. For example, a European televison video standard specifies a frequency of fifty Hertz (50 Hz). That is, the video fields are to be displayed at a rate of fifty fields per second. This televison video rate is not sufficient to prevent noticeable flicker in the television image. To reduce image flicker, the television video rate may be increased to one hundred Hertz (100 Hz) by interpolating additional fields between the original fields in the video image.
Scan rate conversion techniques may be used to convert twenty four Hertz (24 Hz) film to sixty Hertz (60 Hz) video images. Scan rate conversion techniques may also be used to convert thirty Hertz (30 Hz) high definition (HD) camera images to sixty Hertz (60 Hz) video images.
The additional fields needed for scan rate conversion may be acquired by simply repeating the original fields. The preferred method, however, is to use a progressive to interlace conversion using motion compensated interpolation.
There is a need in the art for an improved method of motion compensation to generate interpolated fields that will provide a sharp video image. In particular, there is a need in the art for an improved method of selecting one of two possible choices of a motion compensated pixel for inclusion in a generated field to be interpolated between a previous frame and a next field. There is also a need in the art for an improved method of determining whether the inclusion of the pixel from the previous frame or the inclusion of the pixel from the next field will provide a sharper video image in the generated field.
The present invention comprises an improved method of motion compensation that selects one of two possible choices of a motion compensated pixel for inclusion in a generated field to be interpolated between a previous frame and a next field.
The improved method of the invention comprises the steps of (1) calculating a first correlation value from the values of causal neighbor pixels of a generated field and from the values of corresponding neighbor pixels of a previous frame, (2) comparing the first correlation value with a first threshold value, and (3) setting the value of the pixel to be created within the generated field to be equal to the value of a corresponding pixel of the previous frame if the first correlation value is less than the first threshold value.
The improved method of the invention also comprises the steps of (1) calculating a second correlation value from the values of causal neighbor pixels of a generated field and from the values of corresponding neighbor pixels of a next field, (2) comparing the second correlation value with a second threshold value, and (3) setting the value of the pixel to be created within the generated field to be equal to the value of a corresponding pixel of the next field if the second correlation value is less than the second threshold value.
It is a primary object of the present invention to provide an improved method motion compensated upconversion in a video image upconversion unit of the type that uses motion compensation to generate an interpolated video field using motion vectors.
It is an additional object of the present invention to provide a method for selecting one of two possible choices of a motion compensated pixel for inclusion in a generated field to be interpolated between a previous frame and a next field.
It is another object of the present invention to provide an improved method for selecting a motion compensated pixel that will provide a sharp video image by comparing a calculated correlation value of pixels with a threshold value.
It is also an object of the present invention to provide an improved method for calculating a correlation value of pixels to be used to select a motion compensated pixel that will provide a sharp video image in a generated field to be interpolated between a previous frame and a next field.
It is an additional object of the present invention to provide an improved method for determining a threshold value to be used to select a motion compensated pixel that will provide a sharp video image in a generated field to be interpolated between a previous frame and a next field.
The foregoing has outlined rather broadly the features and technical advantages of the present invention so that those skilled in the art may better understand the Detailed Description of the Invention that follows. Additional features and advantages of the invention will be described hereinafter that form the subject of the claims of the invention. Those skilled in the art should appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the invention in its broadest form.
Before undertaking the Detailed Description of the Invention, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document: the terms xe2x80x9cincludexe2x80x9d and xe2x80x9ccomprisexe2x80x9d and derivatives thereof, mean inclusion without limitation; the term xe2x80x9cor,xe2x80x9d is inclusive, meaning and/or; the phrases xe2x80x9cassociated withxe2x80x9d and xe2x80x9cassociated therewith,xe2x80x9d as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term xe2x80x9ccontroller,xe2x80x9d xe2x80x9cprocessor,xe2x80x9d or xe2x80x9capparatusxe2x80x9d means any device, system or part thereof that controls at least one operation, such a device may be implemented in hardware, firmware or software, or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.